Detection of Assessment Patterns in Ordinary Triadic Conversation

This article outlines a three-party conversation corpus built by the National Institute of Information and Communications Technology (Section 2) and introduces three analyses that contribute to the development of Conversational Informatics: The features of turn-taking in three-person conversation (Section 3); assessment sequential patterns that appeared in the data (Section 4); and shared knowledge and interpersonal relationships between participants observable from the assessment sequences in triadic conversation (Section 5). 1 Significance of Triadic Conversation Analysis The importance of triadic conversation analysis has begun to be recognized in various research disciplines in Japan in recent years. Here are the following attempts of corpus construction, analysis and implementation: 1. Social psychological analysis of “social skills” in triadic conversation [3] [8]: Clarifying the perceived relationships between nonverbal behaviors and expressive dimensions or impressions and rapports. 2. Social agents mediating network communication between two users and the analysis of their social influence power [7] [12]. 3. Multimodal humanoid robots coping with “who to whom” problem in human-human conversation [11]. 4. Building interaction corpus [18] and capturing the dynamics of participation framework [9] from ubiquitous or wearable sensor information in poster presentation environment. 5. Systematics of turn-taking and participation roles in triadic conversation [4] [5]: Fine-grained video analysis of nonverbal behaviors like gaze, gesture and body posture. 2 NICT Three-Party Conversation Corpus 2.1 Recording Design The National Institute of Information and Communications Technology (NICT) has developed and analyzed a three-party conversation corpus. The number of subjects was 45 (15 groups). Each group performed 2 experiments and 30 sets of dialogue were recorded. The average duration required for each experiment was approximately 20 minutes. The speech sound was recorded via DAT and the video image of nonverbal information was via DV. Three DAT decks were utilized: Deck 1 recorded the speech of subjects A+B; D2 recorded that of B+C; and D3 recorded that of C+A on the L & R tracks, respectively. The images of the three subjects were synthesized through the 4-divider and was recorded with one DV deck. The recording by the DV was not for detailed analysis, but for overview of all sounds and images in a dialogue. Each subject entered an individual soundproof compartment to clearly record each speech on an independent track. Therefore, subjects were not placed in a face-to-face situation. We installed two small monitors according to the direction of the input sound so that the members could receive feedback as naturally as possible: A, for example, heard the voices of B and C through the L and R headphones, respectively and could see an image of B in the left monitor and of C in the right (Fig. 1). This environment is like the video telephone conference by three persons, where subjects cannot use eye contact though can obtain speech and nonverbal information. However, the timing of turn-taking seems to be smooth enough. All subjects were university students. Each group consists of 3 members of the same sex (besides one group). The following 4 combinations were prepared: (a) subject A, B and C are friends with each other; (b) A and B are friends + A and C are friends; (c) only A and B are friends; and (d) everyone is a stranger with each other. Each group went through the two tasks consecutively. Experiment 1 was a “photo task” [1], in which subjects are seeking an answer for each of three questions like in Fig. 2, on which all of them can agree. Experiment 2 was a “free topic conversation,” in which subjects could use an optional topic list. 2.2 Theoretical Foci One of the focuses of our analysis was to discover various kinds of linguistic devices that speakers use for audience design [2] Fig. 1. Soundproof compartment Fig. 2. Photo task: These two figures are a. brothers b. husband and wife for four years c. strangers of their utterances that are utilized for turn-taking [15] and assigning participation roles [6]. In the conversation including three or more participants, there is no security or obligation for a current non-speaker to become the next speaker, and determination of the next speaker therefore becomes an indispensable concern for participants. The audience design is linguistic and nonverbal devices by which the next speaker is determined and at the same time the utterance can be understood for all participants. Points we especially tried to clarify for the audience design are: (1) What kind of linguistic devices enable participants to find who, or what kind of participant, should or can become a recipient or the next speaker, besides nonverbal cues like gazes? (2) What kind of shared knowledge and interpersonal relationships between participants are detectable from a variety of audience designs and turn-allocation dependent on them? For (2) above, it is not task-oriented dialogue data in a toy-world but only natural free topical conversation that enables us to uncover ordinary ways of using world knowledge in conversation. 3 Characteristics of Turn-Taking in Triadic Conversation Table 1 shows a list of resources1 for next speaker determination that resulted from the corpus analysis. Table 1 Resources for speaker determination2 I. Nonverbal devices: gaze, etc. II. Linguistic devices: Explicit: vocatives, etc. Tacit: references to participants (personal pronouns or names, their grammatical functions are not vocatives but nominatives, possessives, etc.), choices of addressee honorific/non-honorific particles, selection of specific vocabularies (+III), discourse markers (+IV), etc. III. Use of shared knowledge: Mention to shared episodes, information requests from or giving to someone who does not know them. IV. Use of sequential organization: Second parts of adjacent pairs, continuous questions to the preceding answerer, etc. Unlike explicit devices like vocatives, tacit resources are not specified for speaker selection but have some propositional or modal contents. This means that each of them cannot be used for speaker selection in a context-independent manner, but their functions are implicit triggers for activating shared knowledge or having participants orient to local sequential organization of conversation. Therefore, it is important to clarify their nature of dependency on the context of knowledge or sequential organization.